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I. INTRODUCTION 



The research to be reported herein Is an outgrowth of earlier research 
by Rokeach and his associates (1960) on the interrelations existing 
between personality and the cognitive processes. In the course of this ' 
earlier research we had succeeded in isolating and measuring two distinct 
phases of cognitive functioning; namely, the ability to analyze and the 
ability to synthesize. Analysis was defined, following Witkin 
(1954, 1962), as the ability to separate item from field. Synthesis was 
defined as the ability to integrate new items, elements, ideas, or beliefs 
into new fields, or into new wholes or into new belief systems. It is' 
reasonable to assume that perceptual and conceptual activity in the 
classroom, in the laboratory, in the examination hall, and in everyday 
life involve both the processes of analysis and synthesis to the extent 
that the specific tasks confronting a person require different 
combinations or degrees of the ability to analyze and the ability to 
synthesize. 



In the earlier research we had developed methods for i so 1 at ing \%es e two 
variables and measuring the extent to which perceptual and conceptual 
activity involve the processes of analysis and synthesis. The Doodlebug 
Problem, to be described more fully later on in this report, provided us 
with measures of analysis and synthesis within the framework of solving 
a complicated conceptual problem. The Witkin Embedded-Figures Test and 
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our own modification of the Kohs Block Testi which will also be fully 
described later on, provided us with measures cf perceptual analysis 
and synthesis. 



Although this earlie* work was not specifically concerned with 
educational problems, the following questions having particular relevance 
for education seemed to us to merit further investigation: 

1. To what extent is it meaningful to speak of the cognitive ability 
to analyze and to synthesize? 



2. What are the personality correlates of the cognitive ability to 



analyze and to synthesize? 



3. To what extent are analysis and synthesis correlated or independent 
variables? To the extent that they are independent, they should 
emerge as separate factors in factor analytic studies. 



4. To what extent do current tests of intelligence measure analysis 
and synthesis? 



3. To what extent are analysis and synthesis general rather than specific 
factors? Is a person who is a good synthesizer in one academic 
subject area, for example, also likely to be a good synthesizer in 
another academic subject area? 
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6. To vhat extent can cognitive and personality tests of analysis and 
synthesis predict academic success in various curricula, such as, 
the physical and social sciences, engineering and business, 
literature, the fine arts and mathematics? 

7. To what extent do objective- type and essay-type examinations test 
or tap differentially the ability to analyze and to synthesize? 

8. ' To what extent can the processes of analysis and synthesis be related 

to those of deduction and induction? 

Despite the great amount of theoretical and empirical work available in 
the areas of problem solving (Duncan, 1959; Johnson, 1955), creativity 
(Anderson, 1959; Barron, 1963; Getzels and Jackson, 1962; Guilford, 1956; 
MacKinnon, 1962; McNemar,’ 1964; Rokeach, 1965; Steiner, 1965; Taylor, i960) 
and perception (Allport, 1955; Bartley, 1958) there has been virtually no 
empirical work or theoretical concern with the nature, determinants, 
correlates, and measurement of processes involved in the ability to 
analyze and the ability to synthesize. While there are innumerable 

•V . 

references in the psychological literature and in the Psychological 

< ... 

Abstracts to such topics as productive thinking (Wertheimer, 1945), 

* ; * v* 

critical and logical thinking, concrete and abstract thinking (Goldstein 
and Sheerer, 1941; Harvey, Hunt and Schroder, 1961) and the like, there 
are only occasional references to the abilities of analysis and synthesis 
or to the determinants or correlates of individual differences in such 




abilities. 
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TWO major exceptions should be noted here. Bloom et al. (1956), have 
recognized the importance of analysis and synthesis in the cognitive 
domain by including them in the following listing of six major 
educational objectives : 

1 . Knowledge ’ 

2. Comprehension 

if*/ 

3. Application 

4. Analysis (analysis of elements, of relationships, of organizational 

principles) 

5. Synthesis (Production of a unique ccnnunication, of a plan, or 

proposed set of operations; derivation of a set of 
abstract relationships) 

6. Evaluation 

It is al3o interesting to note that in a volume edited by Brian Simon on 
Psychology in the Soviet Union (1957), the variables of analysis and 

synthesis are mentioned in seven out of 22 chapters written by Soviet 

• > 

"Psychplogist8 . Russian psychologists seem to be greatly interested in 
these processes , particularly in applications which may lead to the 
training of these abilities in children, or to improvements in teaching. 
Despite such concern, however, it does not appear that the Russians have 
succeeded any more than we have in increasing their theoretical under- 
standing of these processes, in measuring them, or in applying them to 
the educational scene. 
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Conceptual and empirical foundations for the present research 

• • 

1. Two personality variables — rigidity and dogmatism — • have been 
found in our earlier research (Rokeach, I960), to be differentially 
correlated with analysis and synthesis in cognition or conception- 
thinking and in perception. Rigidity refers to the resistance to change 
of single beliefs, or sets, or habits, or to the presence of specific 
compulsive or obsessive tendencies within the individual. Dogmatism 
refers, on the other hand, to the resistance to change of tota l systems 
of beliefs . Whereas rigidity is conceived to be a hypothetical property 
of a single belief, or habit, or set, or expectancy, which prevents it 
from changing in the face of objective requirements, dogmatism is 
conceived to be a property of a total system of beliefs, which prevents 
the whole system qua system from changing. For example, we may speak 
of a person as being a dogmatic advocate of psychoanalysis, Marxism, or 
Catholicism, but as being rigid in tying his shoelaces, brushing his 
teeth, or preparing for bed. A rat may be said to behave in a rigid 
(fixated) manner but cannot be said to behave dogmatically. Similarly, a 
mentally retarded, pedantic child may be said to behave rigidly, but not 
dogmatically. To say that a person behaves dogmatically implies that he 
adheres to, espouses, and defends some system or subsystem of beliefs 
(in religion, politics, or science) such that we gain the impression 
that the referent of his behavior is a whole system of ideas rather than 
a single idea. Dogmatism would seem to be a system property in the same 
.way.' that conversion or defection is a system property. When a person 
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changes a single attitude or belief (e.g., about Medicare) we do not say 
that he defects from that attitude and converts to another attitude. 
However, when he changes a ays tern of ideas to another (e.g., from 

Catholicism to Unitarianism, or from Communism to some form of anti* 

• " 

Communism), ve speak of conversion to and defection from. In the same 
way, we should speak of a person as adhering to a system in a dogmatic 

* a i • 

or nondogmatic manner, rather than rigidly or nonrigidly. 

Heedless to say, the variables of rigidity and dogmatism are conceived 
to be continuous rather than discrete. In both our earlier research 
and the present research ve have measured the rigidity variable by means 
of the Gough*Sanford Rigidity Scale and the dogmatism variable by means 
of the Rokeach Dogmatism Scale. Copies of these two scales will be 
found in Appendix A and a fuller discussion of the rationale of these 

two scales will be found in The Open and Closed Mind (Rokeach, 1960). 

. •* 

Subjects scoring high on the Gough- Sanford Rigidity Scale as compared 
with subjects low in rigidity, have been found by Rokeach, McGovney and 
Denny (1955) to take significantly longer to analyze, but not to 
synthesize conceptual problems, as determined by measures of analysis 
and synthesis on the Doodlebug Problem (a problem to be described more 
fully later on). Conversely, subjects scoring high on the Rokeach 
Dogmatism Scale, as compared with subjects scoring low in dogmatism, 
were found to differ significantly in synthesis, but not in analysis. 
These findings are theoretically explained by assuming that personality 
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rigidity, referring as it does to the resistance to change of single 
beliefs, leads to an inability to analyze; i.e., an inability to separate 
an item from its field. Iu short, personality rigidity leads to a 
failure of the ability to analyze or to break down one or more beliefs 
within a total configuration of beliefs. Conversely, we assume that, the 
more dogmatic a person (or, synonymously, the more closed his belief, 
system) the greater the difficulty in synthesizing new systems of beliefs 
because of the resistance to change of (dogmatically held) older systems 
of beliefs, which are in contradiction to the new system. 

2. Evidence from two factor analytic studies by Rokeach and Fruchter 
(1956) and by Fruchter, Rokeach and Novak (1958) indicate that rigidity 
and dogmatism (measured by the Gough- Sanford and by the Rokeach measures) 
are factorially discriminable personality variables. These findings, 
when considered alongside those described in No. 1 above, suggest that 
purely cognitive measures of analysis and synthesis (as obtained, e.g., 
from the Doodlebug Problem) might also turn out to be factorially 
independent and, furthermore, that the personality factors of rigidity 
and dogmatism might have something in common with analysis and synthesis 
at the cognitive level. On the basis of the above stated theoretical 
considerations and empirical findings, it was hypothesized that measures 
of personality rigidity and of the cognitive ability to analyze would 
together form one factor, while a measure of personality dogmatism and 
the cognitive ability to synthesize would together form another factor, 
each factor being independent of or at least discriminable from the other. 
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3. Measures of cognitive analysis and synthesis have been found to 
correlate around .45. Similarly, personality rigidity and dogmatism 
correlate around .45. These findings suggest that even if -igidity and 
analysis should turn out to load on one factor and dogmatism and synthesis 
on another t that such factors would be correlated rather than independent 
factors . 

4. We have found repeatedly in our research with college-student 
samples that analysis and synthesis in problem solving and in perception 
are negligibly related to various tests assumed to measure intelligence. 
Similarly, negligible correlations are found between various intelligence 
tests and the personality measures of rigidity and dogmatism. 

5. Subjects scoring high on the Dogmatism Scale consistently reveal 
greater difficulty than those scoring low, not only in synthesizing ' 
conceptual systems, but also in synthesizing perceptual systems (Levy 
and Rokeach, I960), and aesthetic systems (Mikol, 1960). 

6. Also pertinent to the present research is the work of Witkin and 
his associates (1954, 1962) on field-dependence and independence which 
he conceptualizes in his more recent work in terms of the more pervasive 
process of individual differentiation. The M field- dependent” person is a 
person who, because he is relatively low in individual differentiation, 
lacks the ability to separate item from field and is thus deficient in 
the ability to analyze. In the present research we will use the Witkin 
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Embedded-Figures Test to measure this ability at the level of perceptual 

S 

functioning. 

* 

We regard the findings by Witkin and his associates on field-dependence 
to be closely related to our own empirical findings on individual 
differences in rigidity or in the ability to analyze. When these ' 
findings are considered alongside our empirical findings on the ability 
to synthesize, it seems clear that analysis and synthesis are important 
variables at a “deeper" personality level as veil as the cognitive 
level. We may reasonably assume at the deeper level of personality, the 
existence of individual differences in resistance to change and in 

i 

receptivity to new information, which may result in individual differences 
in the abilities to analyze and to synthesize. 

7. Finally, we have been most impressed by Howard Ehrlich's work 
(1961a, 1961b) on the relation between dogmatism and learning with 
respect to a course in sociology. Ehrlich found that "low dogmatism 
subjects who entered the sociology classroom with a higher level of 
learning, learned more as a result of classroom exposure and retained 
this information to a significantly greater degree than the more dogmatic 
subjects" (1961b, p. 283). What is perhaps even more surprising is that 
Ehrlich found that low dogmatic subjects retained more information about 
sociology not only at the end of the course but also five months after- 
wards and five years afterwards . Ehrlich further found that dogmatism 




"can account for a greater proportion of the variance in the sociology 
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test scores than OSPE (Ohio State Psychological Examination)" (1961a, 
p. 149). 

These findings of Ehrlich suggested that it might be worthwhile to 
explore in considerably greater detail the relationship between selected 
nonintellective variables and academic performance. On the nonintellective 
side we were not only interested in the dogmatism variable but also in 
the rigidity variable. On the academic-performance side we were not only 
interested in Ehrlich's sociology course but in all kinds of academic 
courses, by students who majored and by students who did not major in 
them. 

A more detailed review and discussion of other studies bearing on the 
analysis-synthesis distinction, and on Ehrlich's "dogmatism- learning" 
hypothesis, are reserved for later sections of this report. 
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II. TESTS, PROCEDURE AND SUBJECTS 



All freshmen entering Michigan State University in 1960 and 1961 took 
various aptitude, academic achievement, and personality tests during 
Welcome Week. Included in this battery of tests were the following: 

1. The Michigan State University English Test . The MSU English Test 
is a locally constructed test of 35 objective test items developed by 
Dr. Benjamin B. Hickock in cooperation with the Office of Evaluation 
Services. It has a reliability of .80. The 1958 edition of this test 
measures several aspects of English usage, including capitalization, 
sentence structure, punctuation, grammar, word choice, and the ability 
to organize. 

2. The Michigan State University Reading Test . The MSU Reading Test 
is designed to measure the ability of students to comprehend ideas 
expressed in reading passages that are representative of textual materials 
found in several academic areas. The 1958 edition of this test consists 
of 42 objective items, and was developed by the Office of Evaluation 
Services. It has a reliability of .84. 

3. The College Qualification Test (CQT). The CQT is a nationally- 
used test constructed by the Psychological Corporation, Form C, 1957. 

The CQT Test is broken down into — 
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a. CQT - Verbal (V) 

b. CQT - Information (I) 

c. CQT - Numerical (N) 

d. CQT - Total (T) 

4. The Dogmatism Scale (D) . This is a 40-item scale (Form E) 
originally developed by Rokeach (1956, 1960). The subject agrees or 
disagrees with each item on a seven-point, Likert-type scale, ranging 
from -3 to +3, with the zero point excluded in order to prevent the 
subject from giving a noncommittal answer. In the present research the 
subjects indicated their response to each item on a specially constructed 
IBM form (see Appendix A) so that the response could be machine-scored. 
This scale had been developed on the basis of various theoretical 
considerations to measure individual differences in openness or 
closedness af belief systems. As such it attempts to measure individual 
differences in receptivity to new information and the ability to synthe- 
size or to integrate new information within one's present belief system. 

The Dogmatism Scale differs in one important respect from the well-known 
F Scale. It represents a measure of general authoritarianism, which is 
independent of ideological content, in contrast to the F Scale, which 
represents a measure of rightist or fascistic authoritarianism. Evidence 
for this difference will be found in Rokeach (1956, 1960), in factor 
analytic studies of the total scale scores by Rokeach and Fruchter (1956) , 
and by Fruchter, Rokeach and Novak (1958), and more recently, in a factor 
analytic study of D and F Scale items by Kerlinger and Rokeach (1966). 




13 



A major methodological criticism which has been leveled at both the F 
and Dogmatism Scales is that they are contaminated with an acquiescent 
response set (e.g., Bass, 1955; Christie, Havel and Seidenberg, 1958; 
Couch and Keniston, 1960; Peabody, 1961). All the items in the D and F 
Scales are worded in the same direction so that agreement with the items 
is indicative of authoritarianism or closed-mindedness . It has been 
suggested by many writers that agreement with items on the D and F Scales 
may be indicative of an agreement response bias rather than of 
authoritarianism or closed-mindedness as such since it has been observed 
that some subjects will also agree with the same statements worded in 
an opposite direction. One consequence of the many response set studies 
over the past decade (far too numerous to discuss here) has been an 
increasing reluctance to use the F or Dogmatism Scales unless it has been 
corrected for response bias. 

Rokeach (1963) has pointed out that there are three possible explanations 
for the fact that a person may agree with a statement and agree also with 
its opposite: (1) A person may agree with a scale statement and with 

its opposite because of an acquiescent response set -- a general tendency 
to agree with statements regardless of their content. (2) A person may 
agree with one of the statements because he sees no reason why he 
shouldn't, and he thus tells the truth. He may agree also with the 
reverse statement because he sees a good reason why he shouldn' t tell the 
truth, and thus lies. (3) He may genuinely agree vith both statements 
but through an act of "double-think" or compartmentalization he is not 
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consciously aware of the fact that he holds contradictory beliefs. Many 
persons in a democratic society have been observed to hold both democratic 
and undemocratic ideas, a fact which Gunnar Myrdal has pointed out in his 
classic work. The American nnemiM (1944). 

Rokeach has suggested further that the first interpretation of the 
double agreement phenomenon mentioned above -- acquiescence — is not 
able to account for many of the known findings and Rohrer (1965), 

Block (1965), and Samel'.sort (1964), in recent studies all seem to support 
this conclusion (though for different reasons); namely, that response 
style exerts a trivial influence on responses to various attitude and 
personality tests. Rohrer (1965) after an exhaustive review of 180 

V 

studies in the field concludes that: 

"it does not seem possible that the striking unanimity 
of opinion that various writers have displayed 
concerning the interpretation of this many studies 
could be without any foundation whatsoever; and yet, 
that seems to be the case. The inference that response 
styles are an important variable in personality 
inventories is not warranted on the basis of the 
evidence now available. There is now sufficient 
evidence to conclude that various measures of response 
styles are unrelated when they lack common verbal 
content, from which it follows that there is evidence 
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that present personality inventories are not 
interpretable in terms of personality traits such as 
'acquiescence* (1965, pp. 150- 151). " 

5. The Gouah-Sanford Rigidity Scale . This is a 22- item scale 
measuring rigidity — agreement with items indicating high rigidity. 

This scale is taken from the California Psychological Inventory where it 
is labeled F (Flexibility). The items of this scale were intermingled 
with the items from the Dogmatism Scale and the subjects responded to 
these items on the same specially constructed IBM form using six 
alternatives ranging from -3 to +3 (see Appendix A) . The two factor 
analytic studies mentioned above by Rokeach and Fruchter (1956) and by 
Fruchter, Rokeach and Novak (1958) have shown that while the two variables, 
dogmatism and rigidity, correlate around .45 with one another they 
nevertheless form two factorially distinct factors. 

A total of 798 freshmen were selected for individual testing with the 
Doodlebug Problem, a test which provides measures of conceptual analysis 
and synthesis; with the short form of the Witkin Embedded-Figures Test, 
a test of perceptual analysis; and with our own modification of the Kohs 
Block Test, a test of perceptual synthesis. 

6. The Denny Doodlebug Problem . This test is individually administered 
and takes anywhere from a few minutes to a maximum of 30 minutes, depending 
on the subject. After the subject is seated, the examiner says: 
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Today you are going to be given a newly devised test 
of general intelligence. The problem is not a simple 
one but the solution can be reached by good logical 
analysis. Here is the problem. Read it over carefully. 

(Examiner gives subject a mimeographed sheet which reads as follows) 

The conditions 

Joe Doodlebug is a strange sort of imaginary bug. He 
can and cannot do the following things: 

1. He can jump in only four different directions — 
north, south, east, and west. He cannot jump 
diagonally (e . g . , southeast , northwest , etc . ) . 

2. Once he starts in any direction, i.e., north, 
south, east, or west, he must jump four times in that 
same direction before he can switch to another 
direction. 

3. He can only jump — not crawl, fly, or walk. 

4. He can jump very large distances or very small 
distances — but not less than one inch per jump. 
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5. Joe cannot turn around. 

The situation 

Joe has been jumping all over the place getting some 
exercise when his master places a pile of food three 
feet directly west of him. Joe notices that the pile 
of food is a little larger than he. As soon as Joe 
sees all this food he stops dead in his tracks facing 
north. After all his exercise Joe is very hungry and 
wants to get the food as quickly as possible. Joe 
examines the situation and then says, "Darn it. I'll 
have to jump four times to get the food." 

The problem 

Joe Doodlebug was a smart bug and he was dead right in 
his conclusion. Why do you suppose Joe Doodlebug had 
to take four jumps, no more and no less, to reach the 
food? 

« 

• * * 

The Doodlebug Problem is quite difficult to solve.. With hints from the 
examiner, the average time to solve, as determined by previous research. 




is about 22 minutes. 
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The solution* 

At the moment Joe's master placed the f jod down, Joe had already jumped 
once to the east. He therefore has to jump sideways three more times to 
the east and once, sideways back to the west, thereby landing on top of 
the food. He is now in a positi on to eat the food. 

To arrive at this solution the subject must first overcome three 
currently held beliefs and replace them with three new beliefs, as follows: 

1 . The facing belief . . In our everyday world we assume that one has to 
turn to face the food if one is to be in a position to eat. In the 
Doodlebug Problem, Joe cannot turn to face the food but can land on top 
of it. 

2. The direction belief . In our everyday world we can change our 
direction of movement at will. The subject must come to realize that Joe, 
even though he is forever trapped facing north, can nevertheless change 
his direction of movement by jumping sideways and backwards. 

3. The movement belief . In the everyday world we may change direction 
at will. But Joe cannot change his direction at will because once he 
moves in a particular direction he must make a total of four jumps in 
that direction before he can change to another direction. 



* Needless to say, the solution presented here was not given to the subject 






The total problem solving process can be divided into an analysis and a 
synthesis phase. In the analysis phase the subject is required to 
separate three currently held beliefs from his present belief system in 

which it is embedded and replace them with three new beliefs which are at 

« * . 

variance with the older beliefs. But this is not all. Even after the 
subject is successfully able to accomplish this (with or without the 
examiner’s help) he has not yet solved the problem. He must then organize 
or synthesize the three newly acquired beliefs into a new system. This is 
the synthesis phase of problem solving. 

(After the subject has read the problem, the examiner continues): 

I'd like to ask you to think aloud as you work the 
problem so I can let you know whether you are correct 
or not. You may ask questions as you go along and 
you may refer to the problem at any time. You may use 
the scratch paper in any way you wish. Now let's read 
the problem over together. 

The total time allowed is 30 minutes. For the first 15 minutes the 
subject works continuously regardless of whether he overcomes any of the 
three beliefs by himself. If he overcomes any belief by himself, the 
time taken to do so is recorded. At the end of 15 minutes the 
experimenter asks, "Have you figured it out yet?" 
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If the problem is not yet solved, the examiner gives hints at the end of 
15, 20 and 25 minutes designed to overcome each of the three beliefs. 
Which hint is given depends upon which belief the subject had previously 
overcome by himself. But in general the facing belief is given first, 
the direction belief second, and the movement belief third. Whether or 
not a solution is reached, the session is terminated 30 minutes after 
the problem is given to the subject. 

In the event that the subject overcomes one belief on his own within the 
first 15 minutes, he is given the second hint at the end of 15 minutes 
and the third hint at the end of 20 minutes. In the case where the 
subject overcomes two beliefs within the first 15 minutes by himself, he 
is given the third hint at the end of 15 minutes. This procedure is 
followed for all the subjects without exception. 

The hints are given as needed and as follows: 

1. The facing belief . "I'm going to give you a 
hint. Joe does not have to face the food in order to 
eat it. (Repeat hint.) O.K., I'll give you five 
minutes more." 
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2. The direction belief . "I'll give you another 
hint. Joe can jump sideways and backwards as well as 
£orwards." 

3. The movement belief . "Let's read the problem 
again. (The experimenter and subject reread. the 
problem.) Now here is the last hint. Joe was moving 
east when the food was presented. (Repeat hint.) You 
have five more minutes.” 

At the end of the formal session the subject is given an opportunity to 
comment freely on his reaction’s to the experiment. At this time, too, 
the subject is told the correct solution if he does not already know it, 
is disabused of the idea that the problem is a test of intelligence, and 
is asked not to discuss the experiment with others. 

Following the procedure described above the following measures are 
obtained: 

Analysis measures 

a. Time taken to overcome one belief. 

b. Time taken to overcome two beliefs. 

c. Time taken to overcome all three beliefs. 
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Synthesis measures 



d. Time taken to solve the problem after first belief is overcome. 

e. Time taken to solve the problem after first two beliefs are overcome. 

f. Time taken to solve the problem after all three beliefs are overcome. 

Analysis and Synthesis 



g. Total time to solve the Doodlebug Problem. 



7. The Witkin Embedded-Figures Test -- a test for perceptual analysis . 
The perceptual task used to measure individual differences in perceptual 
analysis was Witkin' 8 Embedded-Figures Test (1950). The full test is made 
up of 24 complex figures and the subject is simply instructed to locate a 
specified simple figure which, he is tcld, is contained within a specified 
complex design. (This test is very similar to a test many people will 
have encountered in their Sunday newspaper supplements wherein a picture 
of a tree is shown along with the assertion that there are many birds 
hidden in it, and the invitation to see how many birds the reader can find 
for himself.) 



The Embedded-Figures * used in this study is the short- form version 

i 

described by Jackson (1*56), consisting of 12 instead of the original 
24 complex figures, with a 3-minute time limit given to locate each simple 
figure contained within the more complex figure. Jackson reports that his 
12-figure version correlates from .96 to .99 with the longer 24-figure test. 
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8. The Modified Kohs Block Design Tet»t -- a test for perceptual synthesis 
This test is a modification of the Kohs Block Design Test as is found on 
the Wechsler Adult Intelligence Scale (1955) or the Golds tein-Scheerer 

i 

Cube Test (1941). In the usual administration of this test the subject 
is shown a series of printed red-and-white designs of varying complexity 
and he is asked to reproduce this printed design with a number of cubes, 
the number varying from four to 16. Each cube has some of its side 
painted white, Rome painted red, and some painted half-red and half-white 
(diagonally) . 

In our own adaptation of this test, which we will henceforth call the 
Perceptual Synthesis Test, the subject is shown a printed design and is 

asked to reproduce it with four blocks. He is then asked to reproduce it 

« • • 

(the same design) once again, but this time the design is to be 

(1) enlarged (built with nine or 16 blocks) and at the same time 

(2) rotated 90 degrees to the left or right (it must not be built first 
and then rotated) and at the same time (3) the colors must be 
interchanged -- red replaces white and white replaces red. All of the 
subject's activity with the blocks takes place with the printed, colored 
design placed in front of the subject in its original position. The 
subject can look at this design all he wants to but is not allowed to 
move it . 

This test is similar in design to the Doodlebug Problem. There are three 
beliefs or sets which the subject has to first overcome and then 
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reintegrate — a size set, a position set, and a color set. By the very 
nature of the task set for the subject he is not able to deal with these 
three sets one at a time* He must deal with all of them simultaneously, 
in an integrative fashion. In contrast with the Witkin Embedded-Figure 
Test, which calls for the analytic separation of item from field, the 

Perceptual Synthesis task requires the simultaneous integration of three * 

. 0 * 

new items into. a new field. 

The Perceptual Synthesis Test has six designs and a five-minute time limit 
is given for each one. In four of them the cubject is required to 
expand the four-block design to a 16-block design; in the remaining two the 
design is expanded from four to nine blocks. - 

As already indicated, a total of 798 subjects served as the subjects in 
the present study. They took the MSU English Test, the MSU Reading Test, 
the CQT, the Dogmatism and Rigidity Scales during Welcome Week under 
conditions of group administration to all entering freshmen. Then, over 
a period of two years subsequent to entering Michigan State University, 
these subjects were selected for individual testing with the Doodlebug 
Problem, the Embedded-Figures Test and the Perceptual Synthesis Test. 

Three hundred twenty-eight of these subjects were tested during the 
1960-61 academic year, and 470 were tested during the 1961-62 academic 
year. The testing took about two hours and the subjects were paid $2 
for their time. These subjects were randomly selected from a larger pool 
of about 2500 entering freshmen from various fields of interest as 
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expressed by the subjects when they entered Michigan State University. 
The number of male and female subjects selected Is shown in Table 1. 

TABLE 1 

Breakdown of 798 Subjects by Field of Interest and Sex 



a 



Field of Interest 


Male 


Female 


Total 


Mathematics, Physical Science, 




• 




and Statistics 


156 


9 


165 


Engineering 


145 


1 


146 


Language and Literature 


13 


107 


120 


Business Administration 


116 


6 


122 


Social Sciences 
(Philosophy, Foreign Studies, 
History, Social Sciences, 
Economics, Political Science, 
Sociology and Psychology) 


64 


124 


188 


No Preference 


23 


34 


57 


Total 


517 


281 


798 



We had originally planned to select approximately equal numbers of 
students from the several fields of interest shown in Table 1. But this 
proved to be impractical due to the fact that many of the subjects we 
selected to represent these fields of interest changed their major as 
their interests changed, or changed to "no preference," etc. It should 
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be obvious, too, that we were not able to get equal numbers of men and 
women for each field of interest, due to the faet that men and women 
are differentially attracted to these fields of interest. 

The subjects who were randomly selected (within the framework of the 
breakdown shown in Table X) were first contacted by letter, and then if 
necessary by phone, to invite then, to the MSU Counseling Center for 
testing. The subjects were told that this testing was for research 
purposes only and that they would be paid for their time at the rate of 
$1 per hour. Approximately 95 per cent of all subjects initially 
contacted responded, made appointments, and were tested. Substitutions 
within the same categories (by major and sex) were made for the remaining 
5 per cent who were not successfully contacted. 

t". 

As already indicated these 798 subjects were individually tested with 
the Doodlebug Problem, the Embedded-Figures Test and the Perceptual 
Synthesis Test over a two-year period. Over this period a total of 13 
examiners were used in all, but the largest majority of our subjects 
(N = 696 for the Doodlebug and N = 527 for the Perceptual Tests) were 
tested by Examiners 1, 2, 3 and 4, 1 and 3 being males and 2 and 4 being 

females. The number of subjects tested on the Doodlebug by these four 



Examiners is shown in Table 2. 
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TABLE 2 

Number of Subjects Tested on the Doodlebug Problem 
by Examiners 1, 2, 3 and 4 



Examiner 


Males 


Females 


Total 


1 


56 


41 


97 


2 


165 


102 


267 


3 


86 


68 


154 


4 


150 


28 


178 


Totals 


457 


239 


696 
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III. THE ABILITY TO ANALYZE AND THE ABILITY TO SYNTHESIZE 



Before presenting the data bearing on the hypothesis that analysis and 
synthesis are different abilities it is necessary to state at the outset 
the conclusion we have come to from our analysis of the present data 
regarding the empirical validity of this hypothesis. The present data, 
because of unforeseen methodological reasons to be described shortly, 
can neither confirm nor disconfirm the hypothesis that analysis and 
synthesis, as operationally measured, are psychologically discriminable 
personality or cognitive processes. 

We could find little or no evidence in this study to support the analysis- 
synthesis distinction. What evidence we have obtained is clearly 
inconsistent with our earlier findings and with those obtained by other 
investigators in the past few years. It was thus necessary to reconcile 
our findings with those of others and to ascertain whether our negative 
findings meant that (a) our specific hypotheses were psychologically 
in.ii.id or whether instead, (b) they could be explained on purely 
methodological grounds. We therefore conducted several additional 
studies not originally planned, which were designed to help us choose 
between alternative (a) and (b). In the course of doing so we obtained 
additional data regarding sex differences in cognitive functioning, 
examiner differences in administering cognitive tests, "contamination" 
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effects, the effects of more simplified ways of scoring the Dogmatism and 
Rigidity Scales, and regarding results obtained from orthogonal factor 
analysis, oblique factor analysis, and Guttman's Smallest Space Analysis 
(Lingoes, 1965). These various findings will be presented and discussed 
in more or less detail in the various sections which follow. 

We have been able to compensate for the unusual methodological problems 
encountered in our study by drawing on the results which have been, in 
the meantime, steadily accumulating in other studies regarding the 
empirical validity of the analysis-synthesis distinction. These studies 
will be reviewed in some detail in a later section. 



A. SEX DIFFERENCES 



It is first necessary to draw attention to a variety of sex differences 
which we discovered in the data. These differenced include the 
Dogmatism Scale, the various measures of analysis and synthesis --on 
both the conceptual and perceptual tests — and the several tests given 
to all incoming freshmen. 

1* Sex differences on dogmatism . Two earlier studies in which the 
Dogmatism Scale had been used, those by Plant (195S) and by Lehmann and 
Ikenberry (1959) have demonstrated the presence of sex differences in 
dogmatism, males scoring significantly higher than females. Table 3 
presents their results along with comparable results obtained in the 
present study showing similar sex differences, and thus confirming the 
results obtained by Plant (1958) and by Lehmann and Ikenberry (1959). 

We are unable to account for these consistent differences except to 
suggest that* they may arise from systematic cultural differences in 
socialization into sex-roles, females being trained to inhibit dogmatic 
expressions of beliefs and attitudes, and males being trained to force- 
fully and aggressively assert such expressions. Another possible 
explanation is that American males are more anxious because the achievement 
demands made upon them within a fiercely competitive society lead 
defensively to a more closed-minded or dogmatic orientation. Earlier 
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research has shown sizable correlations between dogmatism and anxiety 
(Rokeach , 1960; Rokeach and Fruchter, 1956; Fruchter , Rokeach and 
Novak, 1958) but .we have no direct evidence that males are, on the 
average, more anxious than females. 

In contrast to the sex differences in dogmatism we find no consistent 
differences in rigidity. In the present sample the mean rigidity score 
for the men is 97.70 and for the women 97.25 (the standard deviations 
are 23.78 and 23.83 respectively). In the Lehmann- Ikenberry study (1959) 
a small but significant difference is found on the Inventory of Beliefs, 
a measure of stereotypy which is similar to the Rigidity Scale, the males 
being more stereotypic. 

2. Sex differences in cognitive functioning . In addition to sex 

differences in dogmatism we also found systematic sex differences on our 

» 

various individually-administered tests of cognitive functioning, males 
being found to be consistently superior to females in cognitive 
functioning, at least insofar as the measures under consideration are 
concerned. The results f^r the Doodlebug, Embedded-Figures and 
Perceptual Synthesis Tests are shown in Tables 4 and 5. 
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TABLE 4 

Sex Differences in Minutes cn the Various Conceptual Measures 
Obtained from the Doodlebug Test 
(N * 517 Men and 281 Women) 



Variable 

• * i 


Men 

M 


S.D. 


Women 
M S.D. 


t 


P* 


Analysis Measures 

Time taken to overcome 1 belief 


3.81 


3.10 


4.17 


3.26 


1.54 


N.S. 


Time taken to overcome 2 beliefs 


7.41 


3.61 


7.64 


3.55 


.88 


N.S. 


Time taken to overcome 3 beliefs 


10.90 


4.32 


11.26 


4.10 


1.17 


N.S. 


Synthesis Measures 

Time taken to solve after 
first belief was overcome 


13.99 


9.03 


16.02 


9.91 


2.85 


.01 


Time taken to solve after 
first 2 beliefs were overcome 


10.39 


8.27 


12.55 


9.33 


10.29. 


.01 


Time taken to solve after 
ail 3 beliefs were overcome 


6.90 


7.53 


8.93 


8.71 


3.30 

« 


.01 


Total time to solve 


17.80 


9.99 


20.19 


10.72 


3.08 


.01 



* One-tailed test 







TABLE 5 



Sex Differences in Minutes on the Various Perceptual Measures 
Obtained from the Embedded- Figures Test and the Perceptual Synthesis Test 

(N = 517 Men and 281 Women) 



Variable 


Men 

M S.D 


Women 
M S.D. 


t 


P* 


Embedded-Figures (Analysis) 














Total Score 


9.81 


6.92 


12.95 


7.84 


5.63 


.001 


Figure 


1 


.76 


1.11 


.97 


1.34 


2.57 


.01 


Figure 


2 


1/02 


1.29 


1.25 


1.39 


2.20 


.025 


Figure 


3 


.73 


1.21 


1.12 


1.52 


3.71 


.001 


Figure 


4 


3.08 


1.82 


3.63 


1.69 


4.27 


.001 


Figure 


5 


.22 


.60 


.24 


.60 


.45 


N.S. 


Figure 


6 


.21 


.47 


.31 


.62 


2.36 


.01 


Figure 


7 


.31 


.64 


.56 


1.00 


3.79 


.001 


Figure 


8 


1.15 


1 28 


1.39 


1.46 


2.31 


.025 


Figure 


9 


1.04 


1.37 


1.68 


1.69 


5.45 


.001 


Figure 


10 


.63 


.96 


.79 


1.13 


2.01 


.025 


Figure 


11 


.18 


.42 


.21 


.57 


.78 


N.S. 


Figure 


12 


.48 


.84 


.80 


1.15 


4.13 


.001 



Modified Kohs (Synthesis) 
Total Score 


13.99 


5.91 


18.39 


6.03 


9.93 


.001 


Figure 


1 


2.85 


1.43 


3.69 


1.36 


8.24 


.001 


Figure 


2 


2.11 


1.32 


2.95 


1.50 


7.90 


.001 


Figure 


3 


1.83 


1.30 


2.50 


1.48 


6.42 


.001 


Figure 


4 


2.33 


1.47 


3.01 


1.51 


6.15 


.001 


Figure 


5 


2.56 


1.31 


3.29 


1.32 


7.49 


.001 


Figure 


6 


2.30 


1.46 


2.95 


1.53 


5.83 


.001 



* One-tailed test 
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Considering first the sex differences on the Doodlebug Test, it is seen 
from Table 4 that the men took a mean of 17.80 minutes to solve the 
Doodlebug Problem as compared with a mean of 20.19 minutes for the 
women — a difference of over two minutes. Similar differences are 
found without exception on all measures of analysis and synthesis, the 
men in all cases taking less time than the women. The greater superiority 
of men over women in problem solving ability is, of course, consistent 
with other studies reported in the literature. But we hasten to add that 
the sex differences are significant for the synthesis measures but not for 
the analysis measures, suggesting that the generally more superior 
problem solving abilities of men over women reported in the literature may 
be due to the significantly greater ability of men over women to synthesize 
rather than to analyze. But the finding is somewhat paradoxical in the 
sense that these sex differences in synthesizing ability are found despite 
the fact that men are significantly more dogmatic than wome . 

Table 5 shows the mean scores obtained by men and women on the Witkin 
Embedded-Figures Test, a measure of perceptual analysis and the Modified 
Kchs Test, a measure of perceptual synthesis. Once again we note that 

the men are consistently superior to the women on both tasks. Table 5 

* / 

also shows that the results are consistent for all tasks taken 
Individually without exception as well as for the total scores. 
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TABLE 6 

Sex Differences on Various Achievement Tests 
(N = 517 Men and 281 Women) 



Variable 


Men 

M S.D. 


Women 
M S.D. 


t 


P* 


English 


22.74 


5.73 


25.84 


5.06 


7.87 


.001 


Reading 


30.03 


6.05 


30.88 


5.56 


2.00 


.025 


CQT Verbal 


49.31 


12.78 


53.94 


11.74 


5.16 


.001 


CQT Information 


54.57 


8.39 


48.45 


8.77 


6.06 


.001 


CQT Numerical 


37.46 


9.35 


27.98 


9.83 


13.24 


.001 


CQT Total 


141.34 


24.50 

• 


130.37 


24.17 


6.09 


.001 



* Two-tailed test 



The finding of significant sex differences on the Embedoed-Figures Test 
is, of course, consistent with Witkin’s findings. And assuming that the 
Embedded- Figures Test is a measure of perceptual analysis, the sex 
differences found here are in contrast to the absence of such differences 
on conceptual analysis (obtained on the Doodlebug Problem) . At the same 
time it should be noted that the levels of significance are generally 
higher for the perceptual synthesis tasks than for the perceptual analysis 
tasks. All the differences on the perceptual synthesis tasks are without 

,er|c 
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exception, significant beyond the .001 level vhile this is not so for the 
perceptual analysis tasks. It may be noted, for example, that on two of 
the 12 perceptual analysis tasks the differences between men and women 
do not reach statistically significant levels. 

3. Sex differences on other tests . To round out the presentation of 
results bearing cn sex differences, we show in Table 6 the mean scores 
obtained for men and women on various achievements tests taken upon 
entering Michigan State University. 

Consistent with results from numerous other studies, the women are 
significantly superior to the men in English, Reading Comprehension and 
CQT Verbal, the men significantly superior in CQT Information, 

CQT Numerical and CQT Total. These differences in favor of the men are, 
of course, consistent with the findings shown in Tables 4 and 3 showing 
male superiority of cognitive functioning on conceptual and perceptual 
te 1 .^ . 



From all the preceding it is clear that the two sexes cannot be regarded 
as one sample but as two distinctly different ones. We will, therefore, 
report all subsequent data to be presented in this report separately for 




men and women. 
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B. EXAMINER DIFFERENCES 



We found not only sex differences but, more important and with far more 
serious implications for our research program, we also found examiner 
differences on the individually-administered Doodlebug, and Perceptual 
Synthesis Tests. The battery of cognitive tests required about two hours 
for each subject and with 798 subjects, required about 1600 hours of 
individual testing. As already pointed out, such testing was carried 
out over a two-year period and required the service of a relatively 
large number of examiners — eight examiners who were trained to administer 
all three tests and an additional five who were trained to administer 
the Embedded-Figures and Perceptual Synthesis Tests. We examined the 
data for four examiners (two males and two females) who administered the 
great majority of the tests (the remaining examiners being employed 
primarily to facilitate the work of these four examiners) in order to 
ascertain whether differences in performance existed among them. Despite 
the fact that we trained all examiners to administer the tests in a 
standardized manner and despite the fact that early in our testing program 
we conducted quantitative checks to assure that the examiners were not 
producing significantly different results, we nevertheless found, when 
all the data had been collected, statistically significant results among 
the four main examiners. The specific data are shown in Table 7. 



o 
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TABLE 7 

Examiner and Sex Differences in Total Time to Solve the Doodlebug Problem and 

* * # « 

to Complete the Embedded- Figures and Perceptual Synthesis Tests 





A. Total .Time to ! 


Solve the Doodlebug Problem 




V - 


1 




Men 






Women 




! i 


N 


M 


S.D. 


N 


M 


• .S.D. 


Examiner 1 


56 


18.87 


9.42 


41 


27.03 


11.31 


Examiner 2 


165 


16.94 


8.44 


102 


19.09 


10.23 


Examiner 3 


86 


14.64 


8.01 


68 


14.64 


6.48 


Examiner 4 


150 


19.17 


11.23 


28 


22.88 


11.84 




B. Total 


Time on Embedded-Figures 


Test 






Examiner 1 


56 


9.48 


6.08 


41 


14.83 


11.63 


Examiner 2 


165 


8.60 


5.22 


102 . 


12.31 


6.99 


Examiner 3 


86 


8.44 


4.90 


68 


12. H 


6.67 


Examiner 4 


150 


10.88 


7.74 


28 


14.16 


6.96 





C. 


Total 


Time on Perceptual Synthesis Test 




Examiner 1 




56 


14.28 


5.42 


41 


19.42 


6.53 


Examiner 2 


* 


165 


14.12 


6.11 


102 


18.55 


5.99 


Examiner 3 




86 


14.48 


6.02 


68 


18.62 


6.26 


Examiner 4 




150 


12.70 


5.53 


28 


16.76 


6.25 



o 
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In order to fully understand the data shown in Table 7 it should first be 
pointed out that Examiners 1 and 3 are male and Examiners 2 and 4 are 
female graduate assistants. Of even greater relevance is the fact that 
Examiner 1 tested subjects between September, 1960 and June, 1961; 

Examiner 2 tested subjects from January, 196?. to September, 1962; 

Examiner 3 tested subjects from September, 1961 to June, 1962; and 
Examiner 4 tested subjects from December, 1961 tu August, 1962. In other 
words, the data shown in Table 7 represent data obtained in temporal 
sequence — Examiner l's data being obtained first and Examiner 4's data 
being obtained last. 

Before considering the differences in means among the four main examiners, 
let us draw attention once again to the systematic sex differences which 
can be seen in the data. In the case of Examiner 1, for example, the 
mean total time taken by his male subjects to solve the Doodlebug Problem 
is 18.87 minutes and for his female subjects 27.03 minutes -- a difference 
of over 8 minutes. Similar differences in the same direction are observed 
in 3 out of 4 examiners, the means for male and female subjects of 
Examiner 3 being identical. On the Embedded-Figures Test and on the 
Perceptual Synthesis Test, male subjects perform better than female 
subjects for each examiner without exception. These results, of course, 
confirm those shown in Tables 4 and 5. 

Considering now the examiner differences separacel} for males and females, 
it is seen that the means on total Doodlebug scores range from 14.64 
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to 27.03 for the female subjects. The mean Doodlebug scores for 
Examiner l's female subjects are roughly about twice that of Examiner 3's 
female subjects — 27.03 minutes to 14.64 minutes. Analysis of variance 
reveals that the examiner differences are highly significant. 

The comparable examiner differences for the Embedded- Figures Test and for 
the Perceptual Synthesis Test do not appear to be as marked as those for 
the Doodlebug Problem. By analysis of variance we learn that the examiner 
differences are not significant for the Embedded-Figures Test. But the 
examiner differences for the Perceptual Synthesis Test are significant 
beyond the .01 level, although these differences are not nearly as large 
as those obtained for the Doodlebug Problem. 

One possible reason for the large variation among examiners on the 
Doodlebug Problem is that there is a great deal of social communication 
between examiner and subject in this test. The examiner answers all kinds 
of questions put to him by the subject, thus confirming or disconfirming 
the various hypotheses the subject might formulate about the world of 
Joe Doodlebug. Perhaps the large fxaminer differences arise then, from 
the unstandardized manner in which the examiners respond verbally and 
nonverbally despite our training procedures to the subject's remarks, 
queries, etc. 

If this interpretation is correct, it would also account for the smaller 

f 

examiner differences found on the perceptual tests. Once these tasks are 
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set before the subject he is no longer dependent on the examiner, and he 
can proceed on his own to solve the problems. The data show further that 
the examiner differences are systematic rather than random. The male 
subjects of Examiners 2 and 3 solve the Doodlebug Froblem faster than do 
the male subjects of Examiners 1 and 4. The same is true for the female 
subjects of Examiners 2 and 3 as compared with Examiners 1 and 4. Similar 
trends are noted with respect to the Embedded-Figures Test. While these 
consistent trends are not evident with respect to the Perceptual Synthesis 
Test, we nevertheless felt it reasonable to assume that it was the presence 
of personality factors in our examiners which was the source of the 
consistent differences observed. 

On the basis of the data presented in Tables 3 through 7 it was clear that 
the data must be analyzed separately by both sex and examiner. Further 
analysis of examiner differences shows that not only do the means on the 
Doodlebug Problem (and on the Embedded-Figures and Perceptual Synthesis 
Tests) differ from one examiner to the next, but that the relationship's 
among these and other variables also differ. 

For these unforeseen reasons we found it necessary .to depart from our 
original research design. Instead of looking for evidence relevant to our 
main hypotheses regarding the analysis- synthes is distinction in one 
relatively large mass of data obtained from one sample of subjects, it 
was necessary to examine our data separately by sex and by examiner. 
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As already mentioned, we carried out a large number of analyses for 
the total body o£ data, separately by sex, by examiner, and by sex and 
examiner. We found many significant correlations among the various tests 
but the findings were all negative insofar as the analysis-synthesis 
distinction was concerned. We found no. evidence that our various measures 
of analysis were factorially discriminable from our various measures of 
synthesis. We explored and tested many hypotheses which might possibly 
account for these negative findings and which might help to reconcile 
these findings with the positive findings obtained in other studies. 

We finally concluded, on the basis of the data to be presented in the 
next section, that the data obtained on the cognitive tests were 
"contaminated"; that is, that the subjects, unknown to us and contrary to 
our requests for secrecy, were communicating with one another about the 
nature of these cognitive tests. 
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C\ A STUDY OF COMMUNICATION AMONG SUBJECTS: 

TESTS FOR THE INDEPENDENCE OF OBSERVATIONS* 

• • 

i 

As a result of the consistently negative findings with respect to the 
hypotheses already mentioned above, we re-examined the data in an attempt 
to determine the true meaning of the examiner differences. More 
specifically while these differences could have been due to the effect of 
the different examiners, another possibility was that the differences 
were due to the effect of the order in which the subjects took the test. 



If the latter were true, implying communication between subjects already 
examined and prospective subjects, most of the statistical analyses 
relevant to the factorial nature^ of analysis and synthesis would be 
invalid because the observations obtained would not be independent. 

If the reader will turn once again to Table 7 he will note that for both 
male and female subjects the mean "total time to solve the Doodlebug 
Problem" decreases from Examiners 1 to 3 but increases for Examiner 4. 

"V • 

The same trends are observed for the Embedded-Figures Test, except that 
the decreases in mean scores from Examiner 1 to 3 is net as marked as 



* This study was conducted in collaboration with Dr. Rita Zemach of the 
Statistics Department of Michigan State University, who carried out the 
varied analyses described herein. 
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these for the Doodlebug Problem. These particular trends are, however, 
not evident with respect to the Perceptual Synthesis Test. 

If we recall that the data shown for the four examiners are in temporal 
order of testing it does hot appear that the differences among examiners 
are due to -order of testing. 

The means for Examiner 4 do not show the decreases observed from Examiners 
1 to 3 on the Doodlebug and Embedded-Figures Test. The decreases in means 
observed on the Perceptual Synthesis Test do not follow any clear-cut 
temporal order . 

For this reason we initially rejected the hypothesis that the examiner 
differences observed were due, to one extent or another, to order effects. 
But we returned to this hypothesis to examine it more closely when our 
results consistently produced negative findings in the face of various 
confirmations of the analysis-synthesis hypothesis obtained in other 
studies . 

Suppose the observed decreases in mean scores were due to examiner 
differences alone. Then, with the scores arranged in the same order as 
the testis were administered by the examiners, there should be no tendency 
for the scores to decrease within the sample of subjects tested by a 



8 ingle examiner. 
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To test whether this was so, each of the eight sets of subjects, separated 
by examiner and sex, was divided into first-half and last-half, according 
to the order in which the cognitive tests were taken. Under the hypothesis 
that there was no order effect, the mean sccre for the first half of a 
particular sample is just as likely to be lower as higher than the mean 
score for the last half. In addition, the direction of the differences 
for the eight samples should be independent; that is, should fluctuate 
randomly. The question of independence of observations might then be 

r 

answered by testing the null hypothesis that the first-half and second- 
half means do not differ significantly from one another. 

1. The Doodlebug Test-Total Time . Table 8 shows that in every one of 
the eight samples, mean total time to solve the Doodlebug Problem for the 
first half is greater than mean time for the last half. Since this 
result is highly significant (at the .004 level, using a sign test), the 
hypothesis (that the decreases iii mean scores were due to examiner 
differences alone) may be rejected. We conclude therefore, that the 
differences in mean scores between the two halves of a sample are not 
chance fluctuations, but that first-half mean scores tend to be higher 
than last-half mean scores. 



TABLE 8 



Comparison of Means of (a) First Half and (b) Last Half of Samples 
on Total Time to Solve the Doodlebug Problem, by Examiner and Sex 













•• 




Examiner 


Sample 


N 


X 


S.D. 


F 


P 


1 


Male (a) 


28 


19.60 


10.36 


.33 


<*» «■» 


- 


Male (b) 


28 


18.15 


8.50 






Female (a) 


21 


28.49 


11.28 


.65 


-- 


• 


Female (b) 


20 


25.63 


11.43 




2 


Male (a) 


83 


17.02 


8.99 


.02 






Male (b) 


82 


16.86 


7.90 






Female (a) 


51 


20.83 


11.61 


3.00 


'.'.10 




Female (b) 


51 


17.35 


8.40 




3 


Male (a) 


43 


15.85 


8.44 


1.97 






Male (b) 


43 


13.43 


7.47 






Female (a) 


34 


16.20 


7.48 


4.10 


<.05 




Female (b) 


34 


13.09 


4.93 




4 


Male (a) 


75 


23.92 


11.62 


33.96 


<.0005 




Male (b) 


75 


14.22 


8.51 






Female (a) 


14 


26.05 


13.04 


1.96 


- _ 




Female (b) 


14 


19.92 


9.91 
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Since the effect of the order-difference across the first four examiners 
is confounded with the effect of examiner differences, no statistical 
test can be made across examiners. In view of the results of with in- 
examiners differences, however, it is reasonable to assume that the order 
in which the test was taken influenced the scores of the sample as a 
whole. The monotonic pattern of decreasing scores for males and females 
across and within the first three examiners is clearly evident in Table 9. 



TABLE 9 

Doodlebug Total Time 

. Decrease Across and Within First Four Examiners 



Examiner. 




Male 


Female 


1 


First half 
Last half 


19.60 

18.15 


28.49 

25.63 


2 


First half 
Last half 


17.02 

16.86 


20.83 

17.35 


3 


First half 
Last half 


15.85 

13.43 


16.20 

13.09 


4 


First half 
Last half 


23.92 

14.22 


26.05 

19.92 
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In addition to the sign test to establish the trend of the scores, * 
analysis of variance tests were run to determine which of the within- 
sample decreases were statistically significant, even considered apart 
from the general trend. Table 8, which gives the resulting F scores, 
indicates successive increases in the significance of the differences in 
mean scores between first and last half as we proceed from Examiner 1 
to 4. We interpret this to be due to the fact that the opportunity for 
a prospective subject to communicate with a previously examined subject 
increases with the number of subjects examined. In other words, the 
data suggest that the probability of communication increases as a function 
of the number of subjects already examined. 

The most significant difference is found for the male sample of Examiner 4, 
who constitute about 85 percent of the subjects tested by this Examiner. 

The vast majority of these subjects were engineers (see Table 1) who, 
because they were tested relatively late in the two-year period, were 
relatively isolated from those previously tested and at the same time', 
were apparently in frequent communication with one another. The fact 
that these subjects were homogeneous and isolated from those previously 
tested may account for the lack of influence of the first three 
examiners' subjects on th& fourth examiner's subjects. 

•4 

Examiner 4's male sample of 150 subjects was further broken down into 
15 groups of 10 subjects each, arranged in order of testing. Figure 1 
clearly shows a decrease in means as a function of order of testing. 



Total time 



i 
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Order in which test was taken 



Figure 1. Graph showing time to complete Doodlebug Test for male sample 
of Examiner 4, with scores ordered with respect to time test was taken 




(values shown are mean scores for groups of 10) . 
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2. The Conceptual Analysis and Synthesis Scores . The next question 
we asked was whether the order effects observed for the total time to 
solve the Doodlebug Problem was due to order effects only on analysis, or 
only on synthesis, or both. We therefore calculated the mean analysis 
and synthesis scores (mean time taken to overcome three beliefs, and mean 
time taken to solve after the third belief was overcome) for first half 
and second half of samples tested, separately for examiner and sex. The 
results are shown in Table 10. 

An examination of the first-half, last-half mean analysis scores for each 
examiner reveals that in seven of the eight samples (four male and four 
female) there is a decrease in mean time. We find this result significant 
at the .03 level. In the one case of increase (the male sample of 
Examiner 2) the increase is slight; on the other hand, two of the decreases 
are significant beyond the .10 level, one significant beyond the .05 
level, and one highly significant beyond the .0005 level. 

In the case of synthesis, we also find that the tendency for scores to 
decrease from first- to last-half within samples is significant, with 
seven of the eight samples showing a decrease. The one increase (male 
sample of Examiner 1) is negligible. We again point to the highly 
significant decrease in the male sample of Examiner 4. 

3. The Embedded-Figures and Modified Kohs Tests . Comparable analyses 




were carried out to determine whether order effects existed also for the 
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Witkin Embedded-Figures Test and for the Perceptual Synthesis Test. The 
results shown in Table 7 show a decrease in mean scores for both males 
and females across Examiners 1 } 2 and 3 for the Embedded-Figures Test, 
but not for the Perceptual Synthesis Test. The within-examiners ... 
comparisons cf mean scores for first- and last-half, presented in 
Tables 11 and 12, show that scores increased within four of the eight 
samples for the Embedded-Figures Test, and in five of the eight saaples 
for the Perceptual Synthesis Test. Clearly there is no indication of 
any trend within examiner samples for either 'test. Furthermore, the one 
highly significant change in the Perceptual Synthesis scores, for the 
female sample of Examiner 3, involves an increase. For both tests, the 
male sample of Examiner 4 shows unmistakable uniformity, in glaring 
contrast: to the Doodlebug scores. 



TABLE 11 



Comparison of Means of (a) First halfand 

1 % 

on Witkin Embedded-Figures Test, by 



(b) Last half of Samples 
Examiner and Sex 



Examiner • 


Sample 


Kean 


S.D. 


F 


P 


1 


Male (a) 


9.13 


6.68 


.18 






Sale (b) 


9.84 


5.52 






Female (a) 


13.85 


12.21 


.28 






Female (b) 


15.78 


11.26 




2 


Kale (a) 


8.75 


5.24 


.12 






Kale (b) 


8.46 


5.22 






Female (a) 


13.47 


6.62 


2.86 


.10 




Female (b) 


11.16 


7.21 


3 


Kale (a) 
Kale (b) 


8.79 

8.09 


4.78 

5.04 


.44 


— 




Female (a) 
Female (b) 


11.90 

12.33 


6.26 

7.14 


.07 


— 


4 


Hale (a) 


10.90 


6.72 


.00 






Male (b) 


10.88 


8.66 






Female (a) 
Female (b) 


13.67 

14.23 


6.93 

7.44 


.04 


— 
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TABLE 12 



Comparison of Means of (a) First half and (b) Last half of Samples 
on Perceptual Synthesis Test, by Examiner and Sex 



Examiner 


Sample 


Mean 


S.D. 


F 


P 


1 


Male (a) 
Male (b) 


13.80 

14.76 


5.20 

5.69 


.43 


— 




Female (a) 
Female (b) 


18.69 

20.11 


6.26 

6.86 


.48 


— 


2 


Male (a) 
Male (b) 


13.93 

14.32 


6.13 

6.13 


.17 


— 




Female (a) 
Female (b) 


18.86 

13.25 


5.69 

6.33 


: 27 


— 


3 


. Male (a) 
Male (b) 


14.74 

14.21 


S.6S 

6.42 


.17 






Female (a) 
Female (b) 


16.47 

20.77 


6.26 

S.56 


8.98 


.005 


4 


Male (a) 
Male (b) 


12.83 

12»64 


5.71 

5.36 


.04 


¥ 

* 




Female (a) 
Female (b) 


15.36 

18.03 


5.81 

6.65 


1.28 


— 
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In order to assess not only whether communication had a significant 
effect but also how much of an effect, correlations were obtained for 
each of the eight examiner-by-sex samples, between the order in which 
the test was taken and the scores for analysis, synthesis, total . 
Doodlebug Test solution, Witkin Embedded-Figures Test and Modified Kohs 
Test. These results are presented in Table 13. In the case of total 
time, analysis, and synthesis on the Doodlebug Problem, five of the eight 



TABLE 13 

Correlation Between Order in Which Subjects Took Tests 

• •0 m 

and Various Test Scores 



Doodlebug.. Problem 



Examiner 


Sample 


N 


Total Time 


Analysis 


Synthesis 


Witkin 


Kohs 


1 


Male 


56 


-.04 


-.28* 


.07 


-.05 


.03 


1 


Female 


41 


1 

• 

© 

00 


-.11 


-.06 


cOS 


.04 


2 


Male 


165 


-.06 


.01 


-.08 


-.07 


.02 


2 


Female 


162 


-.20* 


-.13 


-.19* 


-.27** 


-.18 


3 


Male 


86 


-.34** 


-.36** 


-.21* 


-.06 


.02 


3 


Female 


68 


-.30** 


-.25* 


-.24* 


.02 


.32** 


4 


Male 


150 


-.38** 


-.48** 


-.27** 


-.03 


-.02 


4 


Female 


28 


-.43* 


-.SO** 


-.36* 


-.22 


-.04 



* Pv05 
** px.Ol 




correlations are significant beyond the .05 level on each variable. It is 
seen also that the correlations are significant for all these three 
variables for Examiners 3 and 4. In all, 15 of the 24 Doodlebug 
correlations are statistically significant. By contrast, only two out 
of the 16 correlations obtained for the Embedded-Figures Test and Modified 
Kohs Test are significant, and these two correlations are in opposite 
directions. 

In view of the results presented above, the conclusion is unavoidable 
that communication occurred among the subjects at the very least with 
respect to the Doodlebug Test, and that communication affected the results 
obtained on analysis, synthesis, and total time to solve the Doodlebug 
Problem. 

One alternative explanation which must be considered is that the examiners 
became more experienced as they continued to test successive subjects and 
as a result, subjects' performance Improved over time. This explanation 
seems unlikely for two reasons: (a) it would not account for the 

observed improvement in problem solving performance on the Doodlebug 
across Examiner 1 to 3; (b) it would not account for the lack of 

improvement of performance within and between examiners on the Embedded- 
Figures Test and on the Perceptual Synthesis Test. 

It should be added, however, that while the finding of significant 
differences in performance related to crder, and of significant 
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correlations between order and performance strongly indicate comnunication 
among subjects, the finding of no differences or zero correlations is not 
necessarily indicative of independence of data or of the absence of 
communication. It is quite possible, and we suspect it is highly likely, 
that the subjects communicated with one another not only about the 

Doodlebug Problem but also about the perceptual tasks. But the Doodlebug 

/ 

Problem is intrinsically more interesting and, it is relatively easy to 
communicate the specific hints (beliefs) and the solution. It is far 
more difficult to communicate the solution or aspects of the solution on 
the various items comprising the Embedded- Figures and Perceptual Synthesis 
Tests. Thus, even though we did not find evidence of improvement of the 
perceptual tasks, we are extremely reluctant to conclude that communication 
among subjects was restricted only to the Doodlebug Problem. 

A simple method for assessing independence of data . A wide-spread 
practice in psychological research with individual subjects, whether it 
involves the administration of some test or the manipulation of some 
experimental variable, is to ask the subject before dismissing him, not 
to discuss the nature of the test or experiment with others. As the 
subject is pledged to secrecy prior to dismissal, it is often explained to 
him that the scientific validity of the data obtained hinges on his 
cooperation in maintaining "security. 11 Many investigators mention in the 
procedure section of their report thctt they carried out this essential 
precaution; they then proceed to discuss their results on the assumption 
that their subjects did, in fact, cooperate as requested. 
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The present research is no exception to this wide-spread practice. We 
too, pledged our subjects to secrecy. We "checked up" on each subject 
who appeared for testing by being on the lookout for unusual behavior 
which might indicate that he had seen, heard of, or discussed the tests 
with others before coming in for testing. We routinely asked the subjects 
at least once during the testing session if he was in any vay familiar’ 
with the tests. 

In approximately half a dozen instances (out of about 800 subjects) the 
subject admitted sometime during the testing session, that he had indeed 
heard of one or more of the tests from others. Needless to say, the data 
obtained from these subjects were discarded, and new subjects were 
substituted to take their place. We then proceeded to assume what is 
undoubtedly assumed in hundreds of other investigations; namely, that 
except for the data of those few who admitted prior communication, the 
data for these remaining subjects are independent of such communication 
and thus valid. 

Accepting this assumption implicitly, we proceeded with a variety of 
analyses designed to test our major hypotheses, and when these hypotheses 
failed to be confirmed, we proceeded with many other analyses in the hope 
that these would help us interpret our negative results as indicating 
either invalid hypotheses or a methodological flaw in the research design. 
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The results described in this section strongly indicate that the subjects 
were communicating with one another and that the probability o£ such 
communication seemed to increase with time. We discovered this fact only 
after formulating, testing and discarding many other hypotheses which 
would help us account for the findings — a process enormously costly 
in time and money. 

One methodological implication of the present findings is that it is now 
possible to propose a simple method for testing for communication effects 
in any body of data obtained from individual subjects who are tested 
successively. The investigator should: 

1. keep a record of the dates his subjects 'jere tested, 

2. order the data temporally, 

3. correlate the order of testing with the measures obtained in the 
research. 

Lack of independence of observations would be strongly indicated if such 
correlations are significantly greater than zero. The larger the 

correlations the less the independence of observations. The larger the 

# 

increase in correlation as a function of time of testing the larger the 
increase in interdependance of observations over time. 

a 

It is reasonable to expect that subjects who communicate with one another, 
in violation of requests from the examiner or experimenter, would be 
motivated to hide this fact, and it is thus necessary to find other ways 
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* * V 

of uncovering evidence of such communication. We have her? proposed a 
simple method for assessing such effects requiring nothing more than the 
calculation of a correlation coefficient between order-of* testing and 
scores obtained. In view of the simplicity of this method, it would not 
be unreasonable to suggest that all studies in which individual subjects 
are tested successively, should routinely report this correlation, in 
the same way that means, standard deviations and reliabilities are 
routinely reported. In view of the unexpected findings reported here, 
the mere assertion by an investigator that his subjects were "pledged to 
secrecy" (or other words to this effect) should no longer be accepted at 
face value to mean that independence of observations is assured. 

Whether a lack of independence of observation is due to communication 
among subjects, or improvements among examiners, or systematic changes 
in the subjects being tested is a matter of careful inference, requiring 
additional data. But regardless of cause, the present method should 
assess any set of data for independence. 
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D. RESULTS CONCERNING THE ANALYSIS- SYNTHESIS HYPOTHESIS 

As already stated, the data obtained in the present research from the 
individually-administered cognitive tests do not provide support for the 
hypothesis that the ability to analyze and the ability to synthesize are 
discriminabiy different abilities. In the previous section we have 
presented data suggesting that the cognitive measures of analysis and 
synth^is are probably invalid because of communication among subjects, 
thus providing us with a purely methodological basis for explaining these 
negative findings. We have presented these methodological findings first, 
rather than the substantive findings, in order to spare the interested 
reader from working his way through a large body of data, only to discover 
in the end that they are at least in part methodologically suspect. 

In this section we will describe in more detail the various kinds of 
analyses carried out to test the analysis-synthesis hypothesis and the 
results obtained. These results, while negative insofar as the analysis- 
synthesis hypothesis is concerned, are nevertheless interesting for two 
reasons: many significant correlations were found among the various 

tests, and the results throw further light on the way in which 
communication among subjects may have affected the substantive results. 
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In line with the data already presented showing sex and examiner 
differences on the Dogmatism Scale and on the individual ly -adminis ter ed 
cognitive tests, we typically analyzed our data (a) separately for males 
and females further subdivided by examiner, (b) separately for the total 
group of males and females, and (c) for the total sample of 798 subjects. 

1. CORRELATIONAL DATA 

We first looked at the correlations among the various personality 
(Rigidity and Dogmatism) and cognitive tests (Doodlebug, Embedded- Figures , 
Modified Kohs) to see if the average correlations within tests assumed to 
measure analysis, or assumed to measure synthesis, were larger than the 
average correlations between measures of analysis and synthes :1s. If our 
assumptions are valid (see Introduction) we would expect scores on the 
Rigidity Scale, on the analysis measures of the Doodlebug Problem and on 
the Embedded-Figured Test to correlate r Ignif icantly with one another; 
conversely, we would expect scores on the Dogmatism Scale, on the 
synthesis measures of the Doodlebug Problem and on the Modified Kohs Test 
to correlate significantly with one another. Furthermore, we expected to 
find that the cluster of variables assumed to measure analysis would not 
correlate significantly or would, at least, correlate less with the 
cluster of variables assumed to measure synthesis. The data concerning 
the intercorrelations among the analysis measures are shown in Table 14. 
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The following points should be noted in Table 14: 

# * • 

1. Quite a few of the correlations are significant beyond the .05 or .01 
levels. Of the 77 correlations shown 29 are significant, 11 beyond the 
.05 level and 18 beyond the .01 level.* 

2. But the results do not seem consistent from one examiner to another 
or for males and females. Examiner 1, 2, 3 and 4 yield, respectively, 

6, 3, 1 and 5 significant correlations and these are not necessarily for 
the same pairs of variables. 

» i ■ • • 

3. In general, the correlations between the personality rigidity 
(as measured by the Gough-Sanford Rigidity Scale) and the analysis 
measures on the Doodlebug (the correlations between variable 2 and 
variables 3, 4 and 5) are lower than the correlations between the 
Doodlebug measures of analysis (variables 3, 4 and 5) and the Witkin 
Embedded-Figures Test (variable 38). Similarly, we note consistently low 
and generally nonsignificant correlations between Rigidity Scale scores 



* Not all the intercorrelations among analysis measures are shown in 
Table 14. We do not show the intercorrelations among those measures on 
the Doodlebug Problem which are not independent; namely, 
variables 3, 4 and 5. 
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